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With the advent of new technologies, methods of blended learning are used in online 
mathematics classrooms to facilitate interactions and provide a richer experience for 
students. This paper analyses data obtained from practising teachers during their 
participation in two postgraduate mathematics courses. We conclude that discussion forum 
interactions are students’ preferred way of online learning. Also, although high levels of 
interaction do not necessarily correlate with success, they are essential for some students to 
persist with difficult content. 

Emerging technologies are increasingly enabling new ways of interacting in both the 
physical and the virtual classroom, allowing higher education instructors to redesign their 
delivery methods. Planning instruction using a variety of delivery modes is often referred to 
as ‘blended learning’ (Bonk & Graham, 2012). In a blended learning environment, learners 
can access course content and activities in a variety of ways, for example, videos, blogs and 
online assessments, while interacting with each other and with the instructor in 
synchronous and/or asynchronous fashion (Holmes, 2005). 

The project we report on in this paper was designed to evaluate the impact of methods 
of blended learning and assessment in a series of online courses, part of a postgraduate 
retraining program for teachers . The student cohort we focused on for this study consisted 
of a group of 60 practising teachers re-training to be high school mathematics teachers. The 
backgrounds of the students were diverse, but generally they were secondary school 
teachers who specialised in areas other than mathematics during their training but found 
that re-training in mathematics would provide better career prospects. 

The project concentrated on two concurrent semester-long courses in the postgraduate 
program. These courses had traditionally been taught online asynchronously by sending 
students material to work on and asking to submit two written assignments and sit a final 
exam. Students were able to communicate via email and online discussion boards with the 
university educators to ask questions and request feedback. With this project we aimed to 
provide a wider range of blended learning experiences for students, including frequent 
targeted online assessment, which we thought could steer the asynchronous nature of the 
online teaching of these courses from a traditional self-directed approach to one in which 
interaction with the university educators and other students was more akin to face-to-face. 
The ultimate goal of the project was to ascertain which factors contribute to a quality 
learning environment in online mathematics teaching. In this paper we aimed to answer the 
following research question: 

What factors do postgraduate students identify as contributing to a quality online 
learning environment? 

Review of Literature 

The question of how to determine the quality of our teaching in the online environment 
is central to recent theoretical and empirical research (Anderson, 2008; Graham, Cagiltay, 

2014. In J. Anderson, M. Cavanagh & A. Prescott (Eds.). Curriculum in focus: Research guided practice 
(Proceedings of the 37 th annual conference of the Mathematics Education Research Group of 
Australasia) pp. 525-533. Sydney: MERGA. 
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Lim, Craner, & Duffy, 2001). It is often argued that the best way to ascertain the quality of 
online teaching is through interactions occurring online and these interactions have been 
experimentally proven to be associated with increasing student achievement (Bernard et al., 
2009). However, many different instruments and measures for analysing the content of 
online interactions are available, and concerns about their validity and reliability have been 
raised (De Wever, Schellens, Valcke, & Van Keer, 2006). 

The work we present in this paper contributes to the body of literature on online 
teaching of mathematics and builds on Engelbrecht and Harding’s (2005) desirable 
attributes for teaching online mathematics. In mathematics, as in many other disciplines, 
the ease, convenience, cost-effectiveness and asynchronous nature of online interactions 
lend themselves to distance education. However, it is of utmost importance for 
mathematics distance educators to be aware of the different affordances of the online 
environment, in particular, issues such as online computerised assessment may play a 
major role in how the courses are ultimately designed and delivered (Threlfall, Pool, & 
Homer, 2013). 

Methodology 

Mathematics courses are amongst the most difficult to teach online due to issues with 
notation and the highly structured nature of the content. Perhaps, for this reason, the 
teaching of mathematics in online environments has traditionally centred on the content to 
be delivered. To facilitate the delivery of mathematics content this project focused on the 
production of two types of online learning activities. On the one hand, and to enable 
students’ self-regulation and engagement, a series of resources were either specifically 
created for the course or externally sourced from the Internet. Externally sourced tasks 
comprised two open-source text-based mathematics books, and many short videos and 
interactive demonstrations covering most of the topics in the course. Our internally 
produced lessons included a set of highly structured course notes, an interactive blog and 
discussion forums. The interactive blogs were delivered every week, and the forums were 
open to students and frequently involved answers to mathematical problems raised in the 
course notes. Due to the nature of mathematical notation, it was sometimes difficult to 
answer mathematical questions using the interface available with our content management 
system. To solve this problem we often posted answers as pencasts, or interactive 
documents containing both written text and voice. On the other hand, designed to enable 
student direction, we created a series of assessment tasks. All tasks were available to 
students from the beginning of the course. Some tasks were due fortnightly, some had to be 
completed within a very strict timeframe, and some involved giving feedback to other 
students. 

Alongside the fortnightly tasks we created a set of surveys in which we asked students 
to give a brief overview of their views and perceptions of courses, encouraging suggestions 
for improvement. We also incorporated in the surveys a Likert-type frequency scale asking 
about their engagement with the different resources created for the courses. The analysis of 
these surveys was of particular interest, as in previous iterations of these courses students 
had submitted handwritten mathematical work covering all topics at the end of the 
semester, whereas this time we opted for two different forms of assessment: a timed 
multiple-choice test to check for basic understanding of the topics presented in the 
preceding two weeks, and a more challenging long response question to ensure deep 
understanding of the topics. The second task, different from previous years, was to be typed 
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(and plotted if necessary) with mathematical typing software we provided to all of our 
students prior to the beginning of the course. It was important for us to see if students felt 
that the resources we created helped them with these tasks. 

All data collection occurred in the first semester of 2013. Our university content 
management system allows the running of analytics concerning the frequency of student 
access to the course and the number of posts made on the discussion board. We collected 
this data for the 60 students enrolled in two courses and correlated it with student 
achievement. 

A subsample of students (n=15), gave consent to participate further in the study and 
completed the fortnightly surveys providing us with information about their level of 
engagement with the different technological innovations introduced in the courses. This 
allowed us to conduct a deeper analysis of the learning environment, as opposed to only 
observable and potentially biased interactions between teachers and students. The analysis 
of the discussion board interactions, the assessment tasks, and the survey responses, 
enabled us to assess the quality of the learning environment as it correlates to the 
technological innovations introduced. 

Results 

The data obtained in this project will now be presented in three sections. Firstly we will 
briefly provide a breakdown and analysis of students’ grades, the number of times they 
proactively intervened in classroom discussions and the time spent online in the courses’ 
pages. This analysis was produced using the analytic tool present in the university’s content 
management system, Blackboard . Secondly, we will analyse the data obtained from the 
students who actively participated in the research project by providing a detailed analysis 
of their activities and the comments they made in the surveys. Lastly, the same analysis 
will be provided about the open discussion forums. 

Relationship between Course Grades, Access Time and Frequency of Discussion 
Board Posts 

For the two courses examined in this study, analytics were sourced from Blackboard© 
to reveal the relationship between students’ grades and their time spent within the course 
and discussion board post frequency. Overall 60 students were enrolled in either or both of 
the two courses under consideration. In Course 1 (focussed on introductory calculus) 41 
students were enrolled, and 50 students were enrolled in Course 2 (focussed on higher level 
mathematics topics for the most part not covered in the high school curriculum). Table 1 
displays the descriptive statistics related to course grades, time spent on course and 
frequency of discussion board posts for each course. 


Table 1 

Course grades, time spent on course and discussion board course frequency 



Course 1 (n=41) 

Course 2 (n=50) 

Course grade - mean (sd) 

78.1(10.9) 

81.6(8.8) 

Time spent on course - mean (sd) 

176(146) 

144(92) 

No. of discussion posts - mean (sd) 

9.7(11.8) 

11.0(12.3) 


Students in both courses were invited to participate in the research project by 
completing regular surveys and allowing access to their final grades, discussion and blog 
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comments. Of the sixty (60) unique students enrolled in either or both courses, fifteen (15) 
provided consent to participate further in the study, with eight students completing one or 
more of five regular surveys throughout the semester. T-tests were conducted to compare 
the mean scores of the fifteen consenting students with the remaining enrollees in each 
course for each of the three measures: course grades, time spent on course and discussion 
board post frequency. The t-test results are displayed in Table 2. 


Table 2 

Comparison study participants and non-participants in relation to course grades, time 
spent on course and frequency of discussion posts 



Participant 

students 

Non-participant 

students 

T-tests 


Mean (sd) 

Mean (sd) 


Course grade: Course 1 

81.7(11.8) 

76.9(10.5) 

t(39) = 1.21, 

p = 0.233 

Course grade: Course 2 

85.8(7.0) 

80.1(8.9) 

t(39) = 2.09, 

p =0.043 

Time (hours) spent in 
course: Course 1 

257(188) 

150(122) 

t(39) = 3.04, 

p =0.000 

Time (hours) spent in 
course: Course 2 

190(127) 

128(72) 

t(48) = 2.08, 

p =0.043 

No. of discussion posts: 
Course 1 

22.8(17.8) 

5 .5(3.9) 

t(48) = 2.16, 

p =0.036 

No. of discussion posts: 
Course 2 

18.7(15.6) 

8 .3(9.8) 

t(48) = 2.80, 

p =0.007 


The 15 participant students in this study were significantly different, as a group, in 
comparison to the remaining students in several ways. The participating cohort, on average, 
achieved a higher course grade, with the difference being statistically significant in Course 
2. In both courses, the participant students spent significantly more time accessing and 
interacting with the online components of the course and posted significantly more 
discussion board posts than the remaining students. Specifically, in Course 1, the 
participant students, on average, posted four times as many posts as the non-participant 
students, and in Course 2, more than twice the number of posts. Therefore, the self-selected 
students were significantly more likely to engage in the online environment than their 
colleagues in each of the courses and they tended to achieve a higher course grade, 
significantly so in Course 2. 

Given that our participant students appeared to be highly active in the online 
environment, in comparison to their peers, an analysis of their survey responses and 
discussion posts was conducted to determine their perceptions of the effectiveness of the 
various online tools and strategies employed throughout the two courses. 

Analysis of Students ’ Experiences with Blended Learning: Survey Responses 

Eight students also responded to up to five surveys throughout the semester seeking 
information about how frequently and in what manner the students were using particular 
tools: reading the course blog, watching links to external videos, watching instructor 
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developed pencasts, reading and/or participating in discussion forums and using online 
tools to contact and interact with the instructors. The mean scores representing the weekly 
frequency of access for each of the tools are displayed in Table 3. 


Table3 

Students' Time Spent in Different Resources Created for the Courses 



Course 1 

Course 2 

Read the course blog - mean (sd) 

3.7 (0.6) 

1.7 (0.1) 

Watched the external video links provided 
(YouTube, etc.) - mean (sd) 

2.8 (0.2) 

2.1 (0.2) 

Read the discussion forums - mean (sd) 

4.9 (1.1) 

4.6 (1.2) 

Participated in the discussion forums - mean 
(sd) 

2.1 (0.3) 

2.9 (0.8) 

Had contact with the lecturer (directly or 
through the forums) - mean (sd) 

2.2 (0.1) 

1.9 (0.2) 


In both courses the most frequently accessed components was to read the discussion 
forums. In this case, students accessed the discussion forums more than four times each 
week. In Course 1 the blog was the second most accessed resource. On average, students 
contacted the lecturers twice a week, either on the discussion board or via email. 

In general the comments from students participating in the research project about both 
courses were very positive, and most students thoroughly enjoyed their experience. 
However, students expressed opinions in the form of comments in the surveys about parts 
of the courses that they particularly liked. Furthermore, they provided constructive advice 
for improvement in some topics. As discussed in the introduction, to help us frame these 
comments and feedback, we have used the seven categories specified by Engelbretch and 
Harding (2005) about teaching undergraduate mathematics on the internet: 

1. Instructor facilitation: 

In general our students were very happy with the way the instructors facilitated the 
communication in the online environment. They made comments such as “I really like the 
blog, both the informal dialogue from the lecturer and the external video links provided. 
[Instructor 1] did a great job, thank you” (Student 13). 

2. Communication opportunities: 

The discussion forums created for the courses were the main means for students to 
communicate with the instructors and each other. Their reflection shows they found them 
very helpful: “I enjoy the discussion forums as a great way to feel connected” (Student 7). 

3. Collaboration opportunities: 

One of the tasks in the course involved a graded discussion in which students were to 
present their thoughts about a topic and then comment and give advice to other students 
about their work. This form of asynchronous online collaboration elicited a positive 
response from students: “Really enjoyed the interaction with other students in both 
courses” (Student 8). However, students felt that synchronous collaboration and 
communication would have been very beneficial: “In other courses I have completed 
through [the university], there have been scheduled in live chat sessions before major 
assignments. This was helpful” (Student 7). 
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4. Cognitive tools: 

Some students pointed out the fact that the use of interactive books would have been of 
great help: “more interactive course notes, where they are embedded with links to video or 
pencasts” (Student 13). It has become apparent to us that this is an area in the courses 
where we need to place emphasis. 

5. Internet resources: 

Most comments students made were very positive “I enjoy the blog and youtube links 
provided. It makes learning so much more interesting and I felt less in the dark - especially 
when the topics were new to me” (Student 8, different week). Several students, however, 
pointed out that externally sourced internet resources were not enough and suggested to 
“have videos created from the lecturer” (Student 15). 

6. Appropriate interface: 

The university content management system, Blackboard©, and our use of it, was the object 
of many students’ comments. In the earlier surveys students expressed feelings such as: 
“I've had trouble finding my way around in blackboard” (Student 20). Another point they 
argued for was more structure in the discussion forums: “It would be great if there were 
discussion topics for each individual topic/chapter so we knew where to look for specific 
information as sometimes a LOT of time is wasted searching for something or asking a 
question that has actually already been answered” (Student 10). 

7. Online assessment: 

We found discrepancies in students’ preferences as to what the best method of assessment 
was. Although some students preferred the timed online tasks: “For [both courses], I love 
the on-line test component. The feedback is pretty much instant and the long responses are 
marked in a timely manner and the feedback has assisted me in my learning” (Student 8). 
The timed tasks presented challenges for some students, who mentioned issues with the 
length of time allocated for their completion saying they needed “more time multiple 
choice” (Student 8). 

Analysis of Students ’ Experiences with Blended Learning: Discussion Forums 

Over the course of the semester, students in both courses actively participated in online 
discussion forums. In many cases, students provided clues as to how they were 
experiencing the course(s) in relation to all aspects of blended learning. To classify their 
work, we will use again Engelbretch and Harding (2005) categories: 

1 . Instructor facil itation. 

Here our students expressed gratitude for interaction with instructors: “This is great, thank 
you. I appreciate you taking the time to answer this question.”(Student 8). Students also 
expressed appreciation for the instructor-created blog, which provided additional support 
throughout the semester: “I would like to thank you for all of the blogs you have done this 
semester. I know they helped me and I am sure others to understand the topics.” (Student 
15). 

2. Communication opportunities: 

Students often took the opportunity to seek out support from fellow students expressing 
feelings such as “How did everyone find the multiple choice test for [this course]. I don’t 
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know if I like this method of testing. I get a bit flustered. I found it hard. Any comments?” 
(Student 17). 

3. Collaboration opportunities. 

The students appreciated the opportunity to collaborate and assist their peers online. For 
example, “I’ve printed off [a particular document]. [...] Has saved me heaps of time. I’ve 
attached it for you” (Student 7). Over time, students became more willing to share their 
work with others, but some reticence was evident in their comments: “Yes, that confused 
me too. [...]. I am attaching some working out that I have done. (I am a bit nervous putting 
this up and hope I don’t embarrass myself).” (Student 8) 

4. Cognitive tools: 

Most of the comments elicited about cognitive tools appeared when students were asked to 
reflect on their teaching as part of one of the assessment tasks: “Applications such as 
Wolfram Alpha, Geogebra, Efofex and other online content make a teacher’s preparation 
time enhanced by providing quality educational tools. ’’(Student 13). 

5. Internet resources: 

Students sometimes commented in the appropriateness of certain resources: “I like the way 
that Khan Academy often gives more than 1 way to think about a solution.”(Student 6). 

6. Appropriate interface: 

While the participant students generally appreciated the tools available within the learning 
management system, some found it challenging at first: “Sorry if this sounds dumb, but this 
is all new to me really. The last assignment I handed in at uni I had a mac classic and no 
internet.” (Student 13) 

7. Online assessment: 

Students expressed some frustration with the online assessment tools: “Yes, I too found 
this test challenging [...]. Unfortunately in this type of testing, the working is not included, 
which limits the ability to demonstrate some level of understanding; it is either right or 
wrong. Thank goodness for the long response section.”(Student 8). 

Conclusions 

From the results above we can infer that students spent most of their online time 
reading the discussion forums and really valued the insight and feedback they obtained 
from them. External links to videos were also an important part of the learning experience 
for students. Our findings corroborate the usefulness of Engelbrecht and Harding’s (2005) 
framework for mathematics online learning. However, we believe that their classification 
could be extended by using well established pedagogical approaches to improving quality 
teaching (NSW DET, 2003). 

It is important to note that students who responded the surveys were above average in 
terms of their interactions in the discussion forums and their time spent online in 
Blackboard©. This would perhaps indicate that the frequency of use of other resources, 
such as reading the course blog or watching externally sourced videos, is probably higher 
than the rest of the student cohort, as would be their interaction with the lecturers. The 
analysis of data of students who did not actively participate in the research project (i.e. 
those who didn’t do the surveys) indicates success in the course is not necessarily 
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dependent upon high levels of interaction with online tools, however, for some students, 
this interaction is absolutely essential to their levels of persistence with difficult content. 

Further Investigations 

With this project we tried to move from an online asynchronous delivery of 
mathematical content to a blended approach where our interactions with students were 
virtually face to face. To do so, we utilised a range of digital resources available for online 
teaching focusing on creating a Quality Learning Environment. This type of environment is 
one of the dimensions of the Quality Teaching framework (NSW DET, 2003). It is known 
that the appropriate use of innovative technologies for online delivery of courses positively 
enhances the learning experience of students. It has also been shown how an approach to 
teaching using the QT framework can lead to improved results in student achievement and 
satisfaction (Gore, Ladwig, Griffiths, & Amosa, 2007) and is applicable to assessment in 
higher education (see ALTC project- http://www.olt.gov.au/proiect-quality-assessment- 
linking-assessment-uon-2007) . What has not been investigated to date is how the QT 
framework could be used to enhance the quality of eLearning in an online or blended 
environment in higher education. We will further the study in this paper by analysing in 
detail using the QT framework all interactions that occurred during the two courses. 
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